Skip to content

Add atomic action benchmarks#336

Open
skywhite1024 wants to merge 6 commits into
ljd/press_atomic_actionsfrom
ljd/benchmark_atomic_actions
Open

Add atomic action benchmarks#336
skywhite1024 wants to merge 6 commits into
ljd/press_atomic_actionsfrom
ljd/benchmark_atomic_actions

Conversation

@skywhite1024

Copy link
Copy Markdown
Collaborator

Description

This PR adds an atomic-action benchmark suite on top of the ljd/press_atomic_actions branch.

It includes benchmark entry points for move_end_effector, move_joints, pick_up, move_held_object, place, and press, plus an aggregate python -m scripts.benchmark atomic-action runner. Object-conditioned benchmarks cover object presets, initial position cases, pickup approaches, optional replay videos, and markdown reports with the required Time & Memory, Success & Other Metrics, and Leaderboard tables.

The pick/place/move-held-object success metrics use physical replay validation so reports distinguish planning success from actual object motion outcomes.

Dependencies: existing EmbodiChain simulation runtime, DexSim, PyTorch, and benchmark assets.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which improves an existing functionality)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (existing functionality will not work without user modification)
  • Documentation update

Screenshots

N/A

Validation

  • python -m black --check --diff --color ./
  • python -m py_compile scripts/benchmark/*.py scripts/benchmark/atomic_action/*.py
  • python -m pytest tests/sim/atomic_actions/test_actions.py
  • python -m scripts.benchmark atomic-action --help
  • python -m scripts.benchmark.atomic_action.pickup_benchmark --help
  • python -m scripts.benchmark.atomic_action.place_benchmark --help
  • python -m scripts.benchmark.atomic_action.move_held_object_benchmark --help
  • python -m scripts.benchmark.atomic_action.press_benchmark --help
  • Smoke/coverage benchmark runs were validated locally by the requester.

Docs build note: make html is blocked in this local environment by missing pypandoc during docs/scripts/sync_readme.py; this PR does not modify docs relative to the target branch.

Checklist

  • I have run the black . command to format the code base.
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • Dependencies have been updated, if applicable.

@skywhite1024 skywhite1024 added atomic action atomic action related functionality enhancement New feature or request motion gen Things related to motion generation for robot labels Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atomic action atomic action related functionality enhancement New feature or request motion gen Things related to motion generation for robot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant